Homotopy Based Algorithms for ℓ0-Regularized Least-Squares

Sparse signal restoration is usually formulated as the minimization of a quadratic cost function ‖y−Ax‖ 2 , where A is a dictionary and x is an unknown sparse vector. It is well-known that imposing an l0 constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the l0-norm is replaced by the l1-norm. Among the many efficient l1 solvers, the homotopy algorithm minimizes ‖y −Ax‖ 2 + λ‖x‖1 with respect to x for a continuum of λ’s. It is inspired by the piecewise regularity of the l1-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem ‖y −Ax‖ 2 + λ‖x‖0 for a continuum of λ’s and propose two heuristic search algorithms for l0-homotopy. Continuation Single Best Replacement is a forward-backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for l0-minimization at a given λ. The adaptive search of the λ-values is inspired by l1-homotopy. l0 Regularization Path Descent is a more complex algorithm exploiting the structural properties of the l0-regularization path, which is piecewise constant with respect to λ. Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection. This work was carried out in part while C. Soussen was visiting IRCCyN during the academic year 2010-2011 with the financial support of CNRS. C. Soussen and D. Brie are with the Université de Lorraine and CNRS at the Centre de Recherche en Automatique de Nancy (UMR 7039).Campus Sciences, B.P. 70239, F-54506 Vandœuvre-lès-Nancy, France. Tel: (+33)-3 83 59 56 43, Fax: (+33)-3 83 68 44 62. E-mail: [email protected], [email protected]. J. Idier is with L’UNAM Université, Ecole Centrale Nantes and CNRS at the Institut de Recherche en Communications et Cybernétique de Nantes (UMR 6597), 1 rue de la Noë, BP 92101, F-44321 Nantes Cedex 3, France. Tel: (+33)-2 40 37 69 09, Fax: (+33)-2 40 37 69 30. E-mail: [email protected]. J. Duan was with CRAN. He is now with the Department of Biomedical Engineering, Xi’an Jiaotong University. No. 28, Xianning West Road, Xi’an 710049, Shaanxi Province, China. Tel: (+86)-29-82 66 86 68, Fax: (+86)-29 82 66 76 67. E-mail: [email protected]. March 18, 2015 DRAFT